Skip to content

feat(portable): record third-party clients in the pinned capability table - #721

Merged
ScriptedAlchemy merged 6 commits into
mainfrom
host-compat-mechanism
Sep 7, 2026
Merged

feat(portable): record third-party clients in the pinned capability table#721
ScriptedAlchemy merged 6 commits into
mainfrom
host-compat-mechanism

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Why

INSTALL.md and two documentation pages asserted a bare list of Agent Plugins
"native clients" (Codex, VS Code, GitHub Copilot, Kiro, ChatGPT) with no source,
no per-client install command, and no statement of what each client actually
loads. The 21 open host-support issues (#693#714) mostly ask the same question:
what does an existing artifact really give a given agent? That is evidence about
a reader of the artifact, not a new projection — so it belongs in the pinned
capability table the adapters already compile against, not in a second catalog.

What

Every unsupported surface is an explicit dated row, never a silent omission:
hooks are unavailable for all five because Agent Plugins 1.0.0 defines no
hooks and this projection emits no hooks document, and each reason names the
file that client would have needed instead.

Gate

Run on host-compat-mechanism rebased onto current origin/main (6bce18953d),
at tip fba03f7cb9:

  • pnpm build — pass
  • pnpm typecheck — pass
  • pnpm lint — pass (1479 files, 88 rules)
  • pnpm test:unit — 0 failed, 4427 passed
  • pnpm docs:site:build — pass: locale drift 0 failures across 36 page pairs,
    0 broken links / 28329 anchors checked

Deslop

Deslop: claude-opus-5-thinking-high, 4 edits — unexported two internal
constants, removed the duplicate readInstall call from the first draft,
dropped the separate host-compatibility.ts catalog in favor of the existing
table, and reworded two adjacent install sentences that leaned on the deleted
claim.

Self-review

Reviewer: gpt-5.6-sol-max (change-risk-reviewer), two passes. Ten findings, all
addressed in 956f0f6d56:

  1. Component-empty bundles claimed clients load skills/mcp.jsonfixed:
    INSTALL.md now prints, per client, only the discovery paths the built
    bundle carries, and says so when it carries none. Both cases are tested.
  2. Qoder's validator command was rendered as its install command, and an empty
    commands array rendered Install: undefinedfixed: the install
    command is first in the record, and an empty list fails the table.
  3. Tier prose claimed a skills client does not read the manifest, which its
    own degraded manifest row contradicted — fixed: the line reads "loads
    its skill tree only", and a tier is now held to its paths as well as its
    rows (agent-plugins names plugin.json, skills names skills, none
    names nothing).
  4. shadowedBy rendered a per-surface narrowing as whole-plugin precedence —
    fixed: it is manifest precedence only; the .mcp.json narrowing lives in
    each mcp row, and Qoder's unproven manifest-precedence row is gone.
  5. A degraded surface could carry no evidence, and paths could be named with
    no discovery evidence — fixed: both are now build failures, with tests.
  6. Path validation was OS-dependent — fixed: isRelocatablePosixPath, with
    a backslash-path rejection test.
  7. The generated host reference rendered raw JSON, bypassing the validator —
    fixed: it renders clientCompatibilityFrom records, so an invalid record
    fails pnpm docs:site:build. Partially dismissed: no dedicated renderer
    unit test was added — the docs build runs the validator on every record and
    validator rejection is unit-tested.
  8. README.md, docs/framework-mode.md, and both index.mdx locales kept the
    unsourced native-client list — fixed in all four.
  9. Grok's record said local-directory install was undocumented while the pinned
    revision documents ./local-dirfixed: the verbatim quote and a local
    install command.
  10. Antigravity's none tier asserted a refusal its citation does not state —
    fixed: the manifest and skills rows are recorded as explicitly unproven.

Review threads (Codex P2, all five resolved in e94d6c56a2)

  1. Install roles. install.actions is now { role, command } with roles
    install, trust, enable, verify, inspect, remove; the validator
    requires exactly one install action and INSTALL.md prints that one, so
    Qoder's validate can never be published as its install command.
  2. Claims from the plan. installSurfaceEntries receives the paths the host
    projections actually planned; each client line names only its recorded paths
    that the build wrote, and says "This bundle emits none of the paths it reads"
    otherwise. A table-driven test renders all four inventories (no component,
    skills only, MCP only, both).
  3. Grok. Recorded as source: marketplace with
    grok plugin install <marketplace plugin name>; the unproven local recipe is
    gone (asserted absent), and trust is its own action and its own evidence line.
  4. Homepage. Both locales now describe the matrix as compatible, partial, and
    incompatible clients; the unsourced list is also gone from README.md and
    docs/framework-mode.md.
  5. Per-file shadowing. shadowedBy is { path, surfaces }; only a shadow
    that takes manifest renders as replacing the plugin, and a per-surface
    shadow renders as "uses it for mcp and still reads the rest".

Also in that commit: degraded.reason is printed as a sub-bullet per narrowed
surface, and both the install prose and the generated reference state that
reading mcp.json and expanding the reserved placeholders are separate rows.

Follow-up in fba03f7cb9: shadowing files keep their authored order, because a
client that publishes a precedence order among those files publishes it in that
order (Devin: .devin-plugin/plugin.json > .claude-plugin/plugin.json > root).

Self-review round 3 (claude-fable-5-1-thinking-high and gpt-5.6-sol-max, tip d3c5362672)

Both reviewers read the diff against origin/main. Findings and disposition:

  1. Shadows ignored the plan (both). Fixed: clientLine splits recorded
    precedence files into the ones this build wrote and the ones it did not. An
    emitted shadow drops the surfaces it takes from Reads: and renders as fact
    ("This build also writes .mcp.json, which it uses for mcp instead."); a file
    that takes every read surface renders as "this build also writes
    .claude-plugin/plugin.json, which it reads as the plugin instead" with no
    install command. A new test composes claude + portable and asserts exactly
    that for Devin, OpenClaw, and Qoder.
  2. Marketplace wording contradicted Grok's own evidence (both). Fixed: the
    sentence is now "Install (no local-directory install is verified for this
    artifact)", mirrored in both clientsIntro strings.
  3. skills tier said "skill tree only" while Qoder also reads mcp.json
    (both).
    Fixed: the tier renders as "loads the components it recognizes
    without reading the manifest", in the prose and both locales.
  4. Grok's install action omitted --trust (Sol). Fixed: the install action
    is the documented spec verbatim (grok plugin install <name> --trust), and the
    separate trust action is /hooks-trust, the in-session command install
    cannot express.
  5. Grok manifest was degraded on the basis Antigravity's is unavailable
    (Fable).
    Fixed: unavailable, same "unproven" reason. Tier unaffected.
  6. Devin's .mcp.json was modeled as a shadow though both documents are read
    and merged (both).
    Fixed: the shadow is gone; mcp is degraded with the
    collision rule quoted, so the limit renders where it applies.
  7. Qoder mcp and OpenClaw skills were degraded for conditions this
    artifact cannot hit (Sol).
    Fixed: both are supported; Qoder's precedence
    lives in its .mcp.json shadow, and OpenClaw's nested-tree limit cannot reach
    an artifact whose skill roots are always immediate children of skills/.
  8. A loaded surface could omit the path it is loaded from (Sol). Fixed: a new
    validator rule binds manifest/skills/mcp to plugin.json/skills/
    mcp.json through the exported CLIENT_SURFACE_PATHS, which the renderer then
    uses to decide which surfaces a build actually wrote.
  9. Install command and degraded reasons rendered unconditionally (Sol).
    Fixed: both are gated on the client reading something this build wrote — a
    componentless bundle now says "there is nothing to install there".
  10. Leftover "native clients" phrasing inside the table (Fable). Fixed in
    lifecycle.uninstall.reason.
  11. Untested validator rules (both). Added: empty command, local-directory
    install at tier none, non-list shadowedBy, unknown surface in a shadow,
    a loaded surface missing its path, and the shadow-order case.
  12. Alphabetical shadow order discarded documented precedence (Sol). Already
    fixed in fba03f7cb9; authored order is kept.
  13. No rendered-output test for the host reference (Sol) — dismissed.
    website/ has no test pool, and pnpm docs:site:build fails on an invalid
    record because it renders through the same validator. The rendered en and zh
    tables were read after the build and carry the roles, per-surface shadows, and
    degraded reasons.
  14. Per-path evidence attribution (Sol) — dismissed. A single pinned
    paragraph routinely covers several paths at once (a discovery section listing
    skills/ and mcp.json together); splitting quotes per path would fragment
    the evidence without making it stronger. The rule stays "at least one dated
    note whenever any path is named".

@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d3c5362

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
agent-bundle Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T22:59:29.967151Z 4d86159 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d86159843

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/agent-bundle/src/install/surface.ts Outdated
Comment thread packages/agent-bundle/src/install/surface.ts Outdated
Comment thread packages/agent-bundle/src/adapters/capabilities/portable-1.0.0.json Outdated
Comment thread website/docs/en/index.mdx Outdated
Comment thread packages/agent-bundle/src/install/surface.ts Outdated

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Owner-requested review — hold for compatibility/installation corrections

Reviewed 3d435889a47acf3d9a1443cab6535c6c9f00d7c6 through GitHub: changed source, capability records, generated-install/documentation paths, tests and existing review threads. No local build or native-host execution was performed by this reviewer. GitHub currently reports conflicts with main.

The direction is appropriate: document readers of the existing portable artifact using the existing capability table, rather than creating another catalog or one runtime per host. This PR is a documentation/compatibility-record slice, not implementation of all 21 native host integrations.

Required corrections — existing P2 threads are valid

I checked the five open inline findings against this head; they are not stale:

  1. Use the actual install action, not array position. src/install/surface.ts::clientLine renders record.install.commands[0] after Install:. Qoder's first element is qoder plugins validate ...; the install command is second. Give actions a small explicit role or render a correctly labelled sequence. Reordering one record alone leaves the same failure mode for the next client.
  2. Do not assert that optional components exist in this artifact. portableInstructions() renders static records without the emitted plan. portable.ts explicitly marks mcp.json optional, and a valid portable selection need not emit portable skills. Distinguish mandatory format markers from optional component discovery paths; do not treat plugin.json + skills + mcp.json as a universal all-required bundle shape. Either qualify these as generic conditional capabilities or use the existing planned/file inventory when saying "this bundle"—not another filesystem discovery pass.
  3. Do not publish the unproved Grok local-install recipe as established setup. The record itself says the consulted examples only establish marketplace-name installation, yet Install: presents <source> for the emitted directory. Omit it or clearly identify its verified source kind. Keep explicit trust approval separate from a generic copy-paste installation recipe.
  4. Fix the homepage's native-reader claim in both locales. A matrix with agent-plugins, skills, and none cannot describe all its entries as native readers of this format. Name compatible, partial, and incompatible/undetermined cases accurately.
  5. Separate whole-manifest precedence from per-component overrides. clientLine says any shadowedBy path makes the whole root "read as that instead". A dotted MCP file can replace the MCP source while shared skills still load. Preserve which surface is affected; do not turn a component override into total format replacement.

Additional guidance grounded in these same paths

clientLine also omits the reason attached to degraded rows. Keep those limitations visible where installation guidance is printed. For Qoder, recognizing the mcp.json file is not evidence that generated ${PLUGIN_ROOT}/${PLUGIN_DATA} launch values execute correctly when the same record marks placeholder behavior unavailable/unproved. Distinguish document recognition from working executable launch, and documentation-derived evidence from a native smoke test. This does not require blocking this PR on every future host implementation.

The fixes can stay small: correct the record semantics and render them faithfully. No universal discovery interpreter, extra projection registry, or duplicated host runtime is needed. The five records here should not close #693#714 collectively; native codecs, SDK entry points, install ownership, and conformance remain the corresponding host issue's work.

Regression matrix before merge

  • Empty/content-free portable fixture, skills-only, MCP-only, and both components: no fabricated present surfaces and no false mandatory-component claim.
  • Portable plus Claude/native marker: whole-format selection distinguished from MCP-only shadowing.
  • Qoder output includes the real install command; validation remains validation.
  • Unverified source kinds and degraded placeholder/launch caveats remain visible in INSTALL and generated en/zh docs.
  • Actual rendered strings must be asserted, not only the presence of the client name or the old misleading sentence.

Resolve the current conflicts, disposition all five inline threads, and record the repository's required local gates for the updated head. The PR workflows retrieved for this reviewed SHA were still pending/queued; this review does not claim they passed.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on 956f0f6 — preserve the scope reduction and finish the evidence-to-prose fixes

Owner-requested follow-up. I checked the current capability records, src/install/surface.ts, tests/install-surface.test.ts, and website/plugins/generated-reference.ts through GitHub. No build, native-host run, or local test was executed by this reviewer. This is a delta to my earlier review, not a request to redo completed fixes.

Scope decision: Aider and Jules are excluded

The owner has removed Jules (#712) and Aider (#713) from the requirements. Their issue comments explicitly say no implementation or compatibility claim should be added. Replace the PR body's current statement about 21 open host-support issues with the reduced requested scope (19 originally requested integrations after these two removals), or avoid an easily stale count and link only the applicable issues. Do not add Aider/Jules capability rows, adapter work, installation instructions, or acceptance obligations while expanding this table. Historical research can remain historical; it is not an active requirement.

P2: unproven compatibility still renders as a proven rejection

portable-1.0.0.json now correctly says Antigravity manifest handling and subsequent skill discovery are unproven, with no CLI run establishing whether the emitted tree is accepted. But src/install/surface.ts::clientTierSentence() still renders tier none as:

loads nothing from this bundle as published

The generated reference's clientsIntro makes the same unconditional claim. tests/install-surface.test.ts still asserts that text and comments that the host rejects the manifest. Thus the record was corrected but the user-visible conclusion was not.

Render this as no compatibility established by this record, and preserve the dated reason. Distinguish an unverified surface from a demonstrated unsupported format; a conservative support policy is appropriate, but it must not masquerade as observed host behavior. This can be a wording/data-rendering correction within the existing table, not a new compatibility subsystem.

Regression: use the present Antigravity record and assert the generated INSTALL and both reference locales do not claim a tested rejection or that component discovery was observed to fail. Update the test's explanation as well as its expected string.

P2: the abbreviated INSTALL line still drops degraded conditions

clientLine() prints reasons for neither degraded rows nor unavailable/unproven rows; its withheld list only selects unavailable/prohibited surface names. The populated Qoder test currently locks in:

loads its skill tree only. Reads: mcp.json, skills.

That is internally confusing, and it hides the actual restrictions already present in the record: .mcp.json takes precedence over mcp.json, and expansion of the generated ${PLUGIN_ROOT}/${PLUGIN_DATA} values is not established. Merely listing a config file as read is not a claim that the generated local launch will work.

Keep the recorded qualifications in the installation-facing output, or link each abbreviated claim directly to a rendered per-client explanation. Separate the skills support level, MCP-document recognition, and launch compatibility rather than describing them all with only. A portable-plus-Claude fixture should keep the MCP precedence caveat visible, without claiming the entire plugin is replaced by a component override. Check the actual rendered text; the current test deliberately expects the conflicting sentence.

Earlier findings that should not be replayed as unchanged

The Qoder record now puts installation before validation, empty command lists are rejected, the reference consumes the shared validator, whole-manifest shadowing no longer lists .mcp.json, and component-empty fixtures now report absent read paths. Those are real improvements. I have not run the newly documented Grok command, so its documented source evidence must remain distinct from native execution proof.

The remaining corrections belong in this PR and the existing host issues; no duplicate issue is needed. This capability-record slice still does not deliver native hooks/SDK entry points/install ownership for every researched host. Preserve the existing implementation boundaries and the owner's explicit exclusion of #712/#713.

…able

Add an optional `clients` block to the portable capability table: per client,
the artifact paths it reads, the manifests that shadow them, its verbatim
install commands, and a dated row per surface it does and does not load.
One validator over the existing records holds a tier to its rows, and
INSTALL.md plus the generated en/zh host reference print those records
instead of an unsourced list of native clients.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant